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EXECUTIVE  SUMMARY 


A  collaborative  program  was  successfully  conducted  by  Humaware  and  the  Aviation  and 
Missile  Research,  Development,  and  Engineering  Center  (AMRDEC)  (managed  by  U.S.  Army 
International  Technology  Center-Atlantic  (USAITC-A))  between  October  2010  and  February 
2011  to  assess  the  perfonnance  of  the  Constant  False  Alarm  Rate  (CFAR)  Autotrend  dynamic 
alert  detection  technology  as  an  augmentation  to  the  Apache  Modernized  Signal  Processor  Unit 
(MSPU)  fault  detection  technology. 

After  an  initial  set  up  and  optimization  of  the  processing  parameters,  these  results  were 
achieved  with  a  single  set  of  parameters  to  control  the  CFAR  and  Autotrend  processing, 
indicating  a  robustness  that  should  result  in  a  low  overhead  for  data  management. 

The  evaluation  data  set  contained  events  for  all  Condition  Indicators  (CIs)  and  for  a  large 
number  of  modules  for  the  MSPU-equipped  AH-64D  Apache  helicopter.  There  was  one  set  of 
CIs  for  the  Auxiliary  Power  Unit  (APU)  that  clearly  indicated  significant  data  quality  issues,  and 
these  are  being  investigated  by  the  Army. 

The  principal  findings  of  the  assessment  are  that  the  CFAR  Autotrend  technology — 

•  Exceeds  the  fixed  threshold  detection  accuracy. 

•  Provides  a  significantly  earlier  detection  than  the  fixed  thresholds. 

•  Detects  maintenance  events  (material  faults)  that  are  not  identified  by  fixed 
thresholds. 

•  Is  ready  for  use  as  an  engineering  tool  in  its  current  fonn. 

The  performance  was  verified  as  well  as  Component  Removal  and  Repair/Overhaul 
Records  (DA  Form  2410)  permitted  and  the  metrics  were  produced  and  verified  in  accordance 
with  ADS-79B,  as  in  Reference  3. 

The  key  results  for  the  maintenance  events  are  in  the  following  table. 


Summary  of  Results 


Metric 

Fixed 

Threshold 

CFAR  Autotrend 
Best  Case 

CFAR  Autotrend 
Worst  Case 

True  Positives  (TP) 

9 

37 

33 

False  Positives  (FP) 

1 

0 

4 

False  Negatives  (FN) 

27 

0 

0 

(The  best  and  worst  cases  are  produced  by  assigning  the  unresolved  detections  to  be  True 
Positive  (TP) — Best  Case  or  False  Positive  (FP) — Worst  Case.) 

The  high  rate  of  False  Negatives  (FN)  in  the  fixed  threshold  results  demonstrate  that  the 
compromise  between  FP  and  TP  embodied  in  simple  thresholding  technique  is  not  present  in  the 
CFAR  Autotrend  detections.  While  the  FN  rate  has  not  been  proven  for  CFAR  Autotrend  by  this 
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trial,  there  is  clear  potential  for  the  detections  to  be  vcriliably  reliable  enough  for  certifying 
maintenance  credits. 

The  same  results  expressed  as  raw  individual  detection  rates  that  make  up  the 
maintenance  events  for  CFAR  Autotrend  are  in  the  following  table. 


Summary  of  FP  Rate 


Metric 

Best  Case 

Worst  Case 

TP 

378 

367 

FP 

5 

33 

Percent  of  FP 

8 

11 

This  FP  rate  is  within  the  target  set  by  ADS-79B.  In  all  cases,  the  CFAR  Autotrend 
detections  occurred  before  fixed  threshold  detections,  as  quantified  in  the  following  table. 

Summary  of  Time  to  Detection 


CFAR 

Autotrend 

Data  Points 

-17 

-18 

Time  (days) 

-27 

-15 

Summary  of  time  to  detection  is  a  key  result  demonstrating  the  potential  for  the 
prognostics  capability  envisioned  for  the  CBM+  program. 

There  is  evidence  to  show  that  the  techniques  are  sensitive  enough  to  detect  incipient 
defects  and  provide  the  opportunity  to  calibrate  the  detection  so  as  to  discriminate  between  wear 
and  faults  that  compromise  airworthiness.  Accuracy,  identifiability,  and  separability  should  all 
be  improved. 

The  Army  has  optimized  a  set  of  fixed  thresholds  for  the  AH-64D  helicopter.  The  CFAR 
Autotrend  has  a  capability  to  retain  these  thresholds  as  “red”  alerts,  and  the  dynamic 
thresholding  can  replace  the  current  fixed  “yellow”  thresholds.  The  CFAR  Autotrend  processing 
also  has  a  feature  that  allows  fixed  thresholds  to  be  used  during  the  initial  flights  following 
maintenance  to  provide  detection  for  defects  present  on  installation. 

AMRDEC  conducted  an  independent  evaluation  of  the  results  to  verify  the  perfonnance 
and  detennine  whether  or  not  the  software  could  be  utilized  independently  of  the  contractor  by 
the  Anny. 

The  chief  finding  is  that  the  CFAR  Autotrend  technology  is  ready  for  use  as  an 
engineering  tool  in  its  current  form. 
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I.  INTRODUCTION 


The  U.S.  Army  has  performed  a  cooperative  project  with  Humaware  to  assess  the 
effectiveness  of  the  Constant  False  Alann  Rate  (CFAR)  Autotrend  technology  in  correct  and 
early  identification  of  faults  as  diagnosed  by  validated  Condition  Indicator  (Cl)  streams  from  the 
Apache  Modernized  Signal  Processor  Unit  (MSPU),  a  Digital  Source  Collector  (DSC)  for 
aircraft  health  data  [1,2].  This  assessment  was  perfonned  with  participation  of  the  technology 
developer. 

The  objective  of  utilizing  the  CFAR  Auto  trend  technology  on  MSPU  data  is  to  offer  an 
enhanced  defect  detection  capability;  that  is,  improved  True  Positive  (TP),  True  Negative  (TN), 
and  False  Positive  (FP)  metrics  when  compared  to  the  MSPU  static  (fixed)  thresholding 
technique. 

The  benefits  of  the  CFAR  Autotrend  event  detection  technology  are: 

•  Automatic  processing: 

o  Fewer  resources  required  to  manage  the  data 
o  Resources  concentrated  on  diagnostics,  not  data  management 
o  100  percent  alert  detection  of  all  CIs 

•  Very  low  False  Alarm  Rate  (FAR): 

o  ADS-79B  compliant  FAR  of  better  than  10  percent  of  true  alerts  [3] 
o  Many  more  valid  alerts  than  false  alerts 

•  Controlled  alert  rate: 

o  Only  alerts  that  lead  to  actionable  maintenance  reported 
o  Manage  work  load  in  reviewing  alerts  to  support  the  diagnostic  processes  and 
also  manage  the  thresholding  verification  process 

•  Information  on  trends  in  data — Turning  point  in  trend  identified 

•  Improved  sensitivity  to  defects — Earlier  warning  of  unscheduled  maintenance 

•  Switch  On-Stay  On  characteristics  provide  more  robust  inputs  into  an  automated 
diagnostics  process  which  leads  to  more  reliable  diagnostics 

•  Can  be  integrated  with  traditional  (static)  alert  thresholds  set  by  the  design  authority 

The  CFAR  Autotrend  technology  reports  relative  changes,  or  trends,  in  the  Cl  data  streams. 
The  technology  replaces  the  “warning”  alert  levels  in  the  current  threshold  management  system 
for  the  MSPU  Health  and  Usage  Monitoring  System  (HUMS)  data,  but  not  the  alann  levels  set  in 
manuals  or  by  the  design  authority.  For  the  warning  alerts,  the  technology  reconciles  the  trade¬ 
off  between  the  Probability  of  Detection  (PD)  and  FAR  without  compromising  either — 
something  that  cannot  be  achieved  with  static  thresholds. 

A  comprehensive  set  of  management  features  means  that  the  alert  rate,  and  more 
importantly,  the  FP  rate  can  be  controlled.  This  provides  reliable,  robust  alerts  for  diagnostics 
and  enough  sensitivity  to  provide  meaningful  prognostics.  The  features  ensure  that  scarce 
HUMS  resources  are  concentrated  on  diagnostics,  not  data  management. 
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An  evaluation  of  CFAR  Autotrend  was  undertaken  by  Humaware  and  the  Aviation  and 
Missile  Research,  Development,  and  Engineering  Center  (AMRDEC)  under  a  Cooperation 
Agreement  (No.  W91  INF- 10-2-0085),  “CFAR  -  Auto-trend  Dynamic  Alert  Technology  Desk 
Top  Demonstrator  for  AH-64  MSPU  Data  Plus  Support  for  Liaison  with  MoD  for  the  Exchange 
of  CH-47  GenHUMS  Data,”  as  outlined  in  the  following  discussion. 

II.  DISCUSSION 

A.  Potential  Impact  on  U.S.  Army  CBM+  System 

Throughout  the  history  of  the  U.S.  Army  CBM+  program,  static  thresholds  have  been 
used  to  define  caution  and  alert  exceedence  levels  for  vibration-based  diagnostics.  It  has  been 
well  demonstrated  that  these  levels  can  be  inadequate  for  differentiating  between  faulted  and 
healthy  populations  [4].  Separability  confidence,  one  of  the  four  key  attributes  of  vibration 
diagnostics  defined  by  ADS-79B,  must  be  achieved  by  the  vibration  diagnostics  for  successful 
implementation  [3]. 

Good  separability  results  in  faulted  populations  that  are  easy  to  distinguish  from  the 
fleet  to  be  identified  for  maintenance  activity.  The  ADS-79B  requirements  for  separability  state 
that  the  FP  rate  shall  be  less  than  10  percent  depending  on  the  criticality  of  the  failure  [3].  The 
acceptable  False  Negative  (FN)  rate  shall  be  10'6  depending  on  the  criticality  of  the  failure.  The 
Anny  is  evaluating  CFAR  Autotrend  to  detennine  if  its  suite  of  algorithms  will  improve  the  FP 
and  FN  rates  of  the  built-in  diagnostics  on  the  MSPU,  and  will  potentially  extend  the  technology 
to  improve  on  the  diagnostics  included  in  all  the  DSCs  installed  on  aircraft. 

The  result  of  implementing  CFAR  Autotrend  is,  therefore,  twofold.  First,  it  will 
improve  diagnostics  that  do  not  respond  well  to  static  thresholds,  and  second,  it  will  allow  for  a 
transition  to  better  understanding  of  remaining  useful  life  which  leads  into  vibration-based 
prognostics.  A  2010  paper,  authored  by  Wade  and  others,  demonstrated  that  traditional 
vibration-based  gear  CIs  exhibit  significant  overlap  between  healthy  and  faulted  populations,  and 
therefore  the  promises  of  CFAR  Autotrend  could  increase  the  effectiveness  of  the  onboard 
software  significantly  [5], 

B.  Scope  of  Evaluation 

The  Army  has  assembled  a  sample  data  set  for  this  evaluation  that  includes  a  total  of  29 
AH-64  aircraft.  The  aircraft  are  further  subdivided  into  5  64A  and  24  64D  model  aircraft. 
Included  in  the  data  set  are  known  TP,  TN,  and  FP  indications  that  resulted  in  component 
teardown  analyses.  Prior  to  beginning  the  evaluation,  the  Army  did  not  know  that  CFAR 
Autotrend  requires  a  minimum  length  of  data  prior  to  component  failure  in  order  to  identify 
anomalies.  Seven  of  the  cases  intended  to  demonstrate  CFAR  Autotrend’s  ability  to  distinguish 
between  healthy  and  faulted  data  did  not  have  the  required  number  of  data  points  prior  to 
component  removal. 
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C.  Evaluation  Methodology 


The  evaluation  methodology  corresponds  to  that  described  in  ADS-79B  for  Detection 
Algorithm  Development  (DAD)  which  summarized  and  agreed  with  the  project  team,  as  follows 

[3]: 


•  For  the  ground  truth  data  set,  achieve  early  detection  when  compared  to  the 
MSPU  data  without  degrading  current  diagnostic  accuracy,  identifiability, 
detectability;  utilize  the  “batting  average”  method  on  the  Anny’s  Aero  Wiki 
page  for  ground  truth  data. 

•  Quantify  the  difference  in  the  detection  times  between  the  CFAR  Autotrend  and 
the  optimized  classic  fixed  thresholds  for  the  ground  truth  data  set. 

•  Show  CFAR  Autotrend  correctly  identifies  features  not  part  of  ground  truth, 
o  Verify  by  referencing  the  maintenance  actions  reported  in  the  DA  Form 

2410  and  tear  down  reports  where  available, 
o  Identify  trend  or  level  events  not  verifiable  in  the  available  ground  truth 
data. 

ADS-79B  identified  perfonnance  targets  of  10  percent  for  FP  and  FN  alarms  are 
accepted  as  the  target  performance  metrics. 

D.  Program  Description 

The  objective  of  the  program  was  to  provide  a  desktop  demonstrator  of  the  CFAR 
Autotrend  automatic  dynamic  alert  generation  technology  for  AMRDEC  technical  staff  and  then 
collaborate  to  evaluate  the  technology’s  features  and  perfonnance. 

The  CFAR  Autotrend  Software  Evaluation  Program  was  defined  by  a  Perfonnance 
Agreement,  as  outlined  in  the  following: 

•  0  to  3  months: 

o  Both  parties  agree  on  specifications  of  a  demonstration  data  set  for 
acceptance  test  (by  way  of  email  exchange).  AMRDEC  will  supply  the 
AH-64  MSPU  demonstration  data  set  to  Humaware  (by  way  of  the 
USAITC-A  cooperative  agreement  manager). 

o  Humaware  will  analyze  demonstration  data  set  and  set  up  the  CFAR- 
Autotrend  processing  parameters  at  home  United  Kingdom  (UK)  location. 

o  Humaware  will  install,  set  to  work,  and  conduct  acceptance  test  in 
Huntsville. 

o  Humaware  will  provide  training,  and  both  will  agree  on  success  metrics  in 
Huntsville. 
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•  3  to  6  months: 

o  Humaware  will  provide  3  months  support  by  way  of  the  Internet. 

o  AMRDEC  will  conduct  independent  evaluation  of  the  software  and 
provide  progress  reports  by  way  of  the  Internet  (monthly) 

•  At  6  months:  Both  will  conduct  an  evaluation  review  meeting  in  Huntsville. 

•  6  to  9  months:  Humaware  will  write  a  report  analyzing  the  variances  of 
AMRDEC  evaluation  results  and  agreed  standards  (completed  as  part  of  6). 

The  period  of  performance  began  in  October  2010.  The  preliminary  results  with  the 
Connector  and  Autotrend  software  with  user  documentation  were  delivered  in  December  2010. 

AMRDEC  provided  Humaware  with  an  example  MSPU  download  that  was  used  to 
develop  the  connector  software  and  for  software  acceptance  purposes.  AMRDEC  generated  the 
evaluation  data  set  and  forwarded  it  to  Humaware.  AMRDEC  and  Humaware  cooperated  to 
identify  the  components  of  the  data  set,  and  Humaware  produced  a  connector  to  enable  the 
CFAR  Autotrend  to  access  and  process  the  data  and  report  the  results. 

The  evaluation  data  set  was  provided  by  AMRDEC  and  Humaware  using  the  connector 
software  and  produced  a  preliminary  evaluation  fixed  database  of  MSPU  data.  Humaware  then 
defined  a  preliminary  set  of  parameters  that  controlled  the  CFAR  Autotrend  processing. 

In  December  2010,  the  Connector  and  Autotrend  software  were  installed  and  accepted 
at  AMRDEC  in  Huntsville.  The  software  was  used  to  train  AMRDEC  engineers  on  the  use  of 
the  technology  and  form  the  basis  of  the  performance  evaluation. 

During  the  December  2010  meeting,  Humaware  and  AMRDEC  agreed  to  the 
performance  metrics  and  the  performance  standards  to  be  met  for  the  evaluation.  Humaware  and 
AMRDEC  then  conducted  the  performance  evaluation  in  accordance  with  the  agreed  criteria. 

Humaware  optimized  the  parameter  settings  for  the  fixed  database  to  maximize  the 
sensitivity  to  defects,  minimize  the  FN  rate,  and  minimize  the  FP  rate. 

An  evaluation  was  conducted  comparing  the  CFAR  Autotrend  results  with  the  DA 
Form  2410  reports,  where  available,  in  Huntsville,  and  the  preliminary  results  were  reported.  It 
was  clear  that  the  CFAR  Autotrend  was  detecting  other  events.  These  were  identified  and 
compared  to  the  DA  Form  2410  data  and  reported. 

After  the  same  software  was  used  for  the  independent  AMRDEC  evaluation,  a  review 
meeting  was  held  and  the  results  were  reported  at  a  final  review  meeting  on  1 1  February  2011  to 
assess  the  performance  of  the  technology  and  software  in  meeting  the  U.S.  Army’s  requirements 
for  dynamic  alert  processing. 
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E.  Preparation  of  the  Evaluation  Data 

Humaware  developed  a  software  connector  to  download  the  data  set  from  the  Comma- 
Separated  Values  (CSV)  files  provided  by  AMRDEC  into  an  Excel  workbook  to  enable  the  data 
to  be  inspected.  A  connector  was  written  to  interface  the  workbook  to  the  CFAR  Autotrend 
processor. 

AMRDEC  provided  a  data  specification  to  enable  the  data  sets  to  be  identified  and 
filtered  for  analysis  (workbook:  AH-64D  definitions  Dan  Wade  Edit  KP  1  final+1+reference.xls). 
In  order  to  provide  an  ability  to  filter  the  data,  Humaware  provided  an  extension  to  the  decode 
connector  so  that  the  Cl  Identity  (ID)  field  could  be  decomposed  into  Component,  Sensor,  and 
Cl-type.  The  reporting  field  types  became  Aircraft  Type,  Tail  number,  Regime,  Module,  CI-ID, 
Component,  Sensor,  and  CL 

F.  Optimization  of  Parameters 

The  CFAR  and  Autotrend  software  uses  control  parameters  to  optimize  the  detection 
performance.  The  parameters  are  defined  in  terms  of  relatively  identifiable  features  of  the  data 
streams  and  do  not  require  specialized  mathematical  knowledge  to  utilize. 

For  the  CFAR  processing,  the  parameters  are  Param  Name,  Separation,  Maximum  Car 
Length,  Box  Length,  Trend  in  Noise  (TiN)  tolerance  (percent),  M,  and  N. 

The  Primary  Threshold  Exceedance  (PTE)  percent  and  Window  length  controls  the 
primary  threshold  level  and  hence  the  sensitivity  of  the  alerts  to  defects.  This  sensitivity  can  be 
reduced,  if  required,  by  setting  the  conservatism  factor. 

The  FAR  is  controlled  by  setting  the  “M”  and  “N”  factors  for  the  M-out-of-N  (MooN) 
processing.  To  predict  the  resulting  FAR,  there  is  a  FAR  calculator,  as  shown  in  Figure  1. 


Level  Secondary  Threshold  False  Alert  Calculator 


Primary 
Threshold 
Exceedance  (%) 

m 

n 

Secondary 
Threshold 
Exceedance  (%) 

Improvement 

Ratio 

3 

5 

7 

4.85E-07 

6.18E+04 

Calculate 


Figure  1.  False  Alert  Calculator 

This  calculator  determines  whether  or  not  a  target  FAR  is  achieved.  It  is  also  possible 
to  set  parameters  that  determine  whether  the  level  change  is  large  enough  in  amplitude  to  be  a 
discontinuity  or  a  step  change. 

The  Autotrend  process  is  controlled  by  Separation,  Maximum  Car  Length,  Box  Length, 
TiN  Tolerance  (percent),  M,  and  N  parameters. 
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The  Box  and  Maximum  Car  lengths  detennine  the  length  of  trends  that  are  detected. 
When  the  trend  is  detected,  it  is  controlled  by  the  Separation  parameter.  The  TiN  Tolerance  and 
M  and  N  factors  are  used  to  determine  that  only  significant  and  sustained  trends  are  alerted.  This 
ensures  that  the  FAR  for  Autotrend  is  similar  to  the  CFAR  rate. 

The  procedures  for  optimization  of  these  parameters  are  contained  in  Reference  6,  and 
details  of  the  functionality  can  be  found  in  Reference  1 . 

Initially,  to  establish  a  basic  parameter  set,  a  small  sample  of  the  data  was  analyzed  by 
Humaware  using  the  CFAR  Autotrend  analysis  facilities.  This  parameter  set  was  then  applied  to 
the  fixed  evaluation  database  with  the  results  reported  in  Table  1. 


Table  1.  CFAR  Auto  trend  Total  Finds 


Metric 

CFAR  Autotrend 

TP 

480 

FP 

123 

Uncertain 

165 

Invalid  Data 

54 

Total 

822 

The  TP  results  for  the  CFAR  processing  are  reported  in  Table  1.  Examples  shown  in 
Figure  1  demonstrate  that  the  primary  threshold  can  track  rising  (first  chart)  or  falling  (second 
chart)  data  streams  and  are  not  confused  by  random  peaks  in  the  data.  When  the  alert  is 
indicated,  the  indication  remains  set  despite  some  drop-outs  in  the  data.  The  large  number  of 
threshold  value  changes  in  the  charts  demonstrates  the  requirement  for  the  CFAR  dynamic 
threshold  setting  functionality. 
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Figure  2  shows  that  the  Auo trend  processing  can  accurately  identify  trends  and  their 
turning  points,  and  that  the  processing  is  not  confused  by  other  types  of  events,  such  as  level 
changes  or  shallow  trends  that  are  not  diagnostically  useful,  as  shown  in  Figure  3. 


Level  Alert:  00-00151  4  GG  Shaft  (Params:  7;7;6) 


Level  Alert:  00-00138  7  TR  Drive  Shaft  (Params:  7;7;6) 


Figure  2.  Example  Primary  Threshold  Use 
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Trend  Alert:  00-00143  2  JK1  Bearing  (Params:  2.3;60) 


Trend  Alert:  00-00151  11  37  Gear  and  Bearing  (Params:  2.3;60) 


Figure  3.  Example  Trends 

Figure  4  is  typical  of  the  conditions  that  produce  the  FPs  reported  in  Table  1.  The  data 
were  unusually  correlated,  and  a  large  number  of  the  points  appeared  to  be  repeated.  These  data 
randomly  correlate  to  form  wide  pulses.  The  conditions  cause  both  the  CFAR  and  Autotrend 
processing  to  generate  FPs.  If  this  were  typical  data  associated  with  a  Cl  type,  component,  or 
sensor,  the  set  parameters  could  be  adjusted  to  compensate  for  the  correlation.  These  data  types 
occur  sporadically  in  the  data  set,  so  the  FPs  that  result  are  an  irreducible  component  of  the 
performance  of  the  system. 
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Level  Alert:  00-00136  22  57T  Gear  (Params:  7;7;6) 


Trend  Alert:  00-00152  10  22  Gear  and  Bearing  (Params: 

2.3;60) 


Figure  4.  Typical  False  Alarm  Situations 
The  invalid  data  reported  in  Table  1  was  of  the  form  shown  in  Figure  5. 
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Trend  Alert:  00-00130  26  Clutch  Bearing  (Params:  2. 15;60) 


Trend  Alert:  00-00132  8  Bearing  (Params:  2.1 5;60) 


Figure  5.  Example  of  Invalid  Data 

When  tones  correlate  (such  as  those  in  the  previous  figures),  they  generate  FPs  in  the 
Autotrend  data.  This  behavior  is  dominated  by  data  from  the  Auxiliary  Power  Unit  (APU), 
which  is  found  only  sporadically  in  any  other  module.  AMRDEC  agreed  to  investigate  the  data 
quality  of  the  APU  and  also  agreed  that  these  data  sets  were  not  to  be  included  in  the 
performance  analysis.  The  Parameter  Allocation  Table  (PAT)  in  the  CFAR  Autotrend  was  used 
to  switch  off  the  processing  from  this  source. 

The  control  parameters  for  the  CFAR  Autotrend  parameters  were  reviewed  to  optimize 
the  TP  and  FP  metrics. 

There  were  a  large  number  of  multiple  alerts  on  the  Autotrend  in  particular,  as  shown  in 
Figure  6.  The  inflated  number  of  true  alerts  indicated  that  the  box  car  length  (the  number  of 
points  that  constitute  a  trend  by  the  diagnostics  process)  was  too  short. 
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Trend  Alert:  00-00130  15  Tail  Rotor  Take  Off  Shaft  (Params:  2. 15;60) 


Trend  Alert:  00-00130  7  TR  Drive  Shaft  (Params:  2. 15;60) 


2.56 
2.06 

1.56 
1  06 
0.56 


Figure  6.  Example  of  Multiple  Trend  Detections  in  the  Same  Trend 


The  box-length  parameter  was  increased,  as  was  the  TiN  parameter,  to  reduce  the 
number  of  low  rate  trends  that  produced  alerts.  The  MooN  parameters  were  increased  to  remove 
trends  that  were  not  sustained,  therefore  removing  the  spurious  alerts.  There  were  a  number  of 
trends  that  occurred  as  a  result  of  level  changes  not  alerted  by  the  CFAR  processing,  as  shown  in 
Figure  7. 
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Trend  Alert:  00-00151  5  Bearings  (Params:  2. 15;60) 

4.51 
4.01 

3.51 
3.01 

2.51 
2.01 

1.51 

-460  -420  -380  -340  -300  -260  -220  -180  -140  -100  -60  -20 


0.11 
0.09 
0.07 
0.05 
0.03 
0.01 

-594  -554  -514  -474  -434  -394  -354  -314  -274  -234  -194  -154  -114  -74  -34 


Trend  Alert:  00-00130  5  Bearings  (Params:  2. 15;60) 

3.59  - 
3.09  - 

2.59  - 
2.09 

1.59 

-610  -570-530-490-450-410-370-330-290-250-210-170-130  -90  -50  -10 

Figure  7.  Example  of  Trends  that  Should  Be  Level  Alerts 

This  effect  is  caused  by  the  primary  threshold  in  the  CFAR  processing  being  too  high. 
To  correct  this,  the  Primary  Threshold’s  PTE  rate  was  increased  to  make  it  more  sensitive  to  the 
level  changes.  The  secondary  processing  MooN  was  maintained  to  keep  the  FAR  in  excess  of  1 
in  100,000. 


Trend  Alert:  00-00130  4  Output  Shaft  (Params:  2.1 5; 60) 
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The  value  at  which  to  report  a  detection  of  a  level  change  is  set  by  a  conservatism 
factor.  Any  changes  made  to  the  conservatism  factor  parameter  caused  more  TP  alerts  to  be  lost 
than  the  reduction  of  FP  alerts  achieved,  so  it  was  maintained  at  1.  The  discontinuity  or  step 
detector  conservatism  level  was  adjusted  to  remove  ambiguities  between  step  and  level  alerts. 

The  resulting  changes  in  the  parameter  sets  with  the  reasons  for  the  changes  are  shown 
in  Tables  2  and  3. 


Table  2.  Before  and  After  Parameters  for  Auto  trend 


Parameter 

Name 

Separation 

Maximum 
Car  Length 

Box  Length 

TiN  Tolerance 
(%) 

M 

N 

Kptestl 

2.15 

60 

30 

20 

2 

3 

Increased  to 
reduce 
spurious 
trends 

Increased  to 
reduce 
number  of 
split  trends 

Increased  to 
remove  low  rate 
trends 

Increased  to 

remove 

spurious 

trends 

Increased  to 

remove 

spurious 

trends 

kptest2 

2.3 

60 

40 

30 

3 

4 

Table  3.  Before  and  After  Parameters  for  CFAR 


Parameter 

Name 

Primary 
Threshold 
Exceedance  (%) 

Window 

Length 

Level 

Conservatism 

Factor 

M 

N 

Step 

Conservatism 

Factor 

Step 

M 

kptestl 

3 

70 

1 

5 

7 

15 

3 

Increased  to 
increase 

sensitivity  to  level 
changes 

Reduced  to 
better  track  the 
envelope  of 
the  signal 

Increased 
to  maintain 
target  FAR 

Increased  to  reduce 
number  of  alerts 
due  to  pulses  in 
data  set 

kptest4 

7 

60 

1 

6 

7 

22 

3 

These  modifications  to  the  control  parameters  resulted  in  the  performance  as  shown  in 
Table  4. 


Table  4.  Final  Results  After  Parameter  Adjustments 


Metric 

Old 

New 

TP 

480 

367 

FP 

123 

33 

Uncertain 

165 

11 

Invalid  Data 

54 

29 

Total 

822 

440 

The  uncertain  alerts  were  compared  to  DA  Form  2410  data  to  determine  which  were 
true  alerts  and  which  were  false  alerts.  Eleven  alerts  could  not  be  resolved. 

The  resulting  33  FP  alerts  are  randomly  distributed  across  module,  Cl  type,  aircraft,  and 
sensor  and  are,  therefore,  considered  an  irreducible  residual  FP  rate. 
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G.  Results 


The  results  of  the  processing,  as  used  for  the  analysis,  are  contained  in  the  Excel 
Workbook  Evaluation  data  Version  1.2. 3.xls  [7].  There  are  three  worksheets:  MSPU  data, 
CFAR  discovered  data,  and  uncertain  data.  The  uncertain  data  were  the  borderline  CFAR 
Auto  trend  alerts  that  required  DA  Fonn  2410  evaluation  in  order  to  determine  whether  they  were 
true  or  false  alerts.  These  results  were  achieved  with  a  single  CFAR  and  Auto  trend  parameter 
set. 


H.  MSPU  Detected  Defects 

The  evaluation  MSPU  data  set  consisted  of  20  examples.  Of  the  20,  1 1  were  either 
alerted  on  the  component’s  installation  or  insufficient  data  was  provided.  The  CFAR  Autotrend 
technique  identifies  the  relative  changes  in  the  data,  and  since  an  installed  defect  does  not 
produce  a  relative  change  in  the  data,  absolute  techniques  will  still  be  required  to  detect  these 
defects.  Therefore,  they  were  excluded  from  the  analysis.  The  remaining  nine  cases  produced 
the  metrics  as  shown  in  Table  5. 


Table  5.  ADS-79  Metrics  Summary  for  Fixed  Thresholds 


Metric 

Fixed 

Thresholds 

CFAR  Autotrend 

Ground  Truth 

TP 

7 

8 

8 

FP 

1 

0 

TN 

1 

1 

FN 

1 

0 

Current  operational  procedures  affirm  that  a  single  Cl  is  acceptable  to  produce  a  fault 
indication.  Of  the  CFAR  Autotrend  alerts,  only  one  alert  was  a  single  Cl  alert.  The  remaining 
alerts  were  multi-CI,  which  may  improve  the  accuracy,  indenti  flability,  and  separability  of  the 
diagnostic  processing. 

The  CFAR  identified  two  defects  that  were  not  detected  by  the  fixed  thresholds,  which 
is  evidence  of  greater  sensitivity  to  defects. 

The  timing  of  the  alerts  for  CFAR  were,  on  average  (with  Standard  Deviations),  11(16) 
points  or  24  (30)  days  earlier  than  the  fixed  threshold  alerts,  and  the  trend  detections  were  35 
(16)  points  or  34  (5)  days  in  advance  of  the  fixed  threshold  date.  The  large  standard  deviation  in 
the  “days”  sample  interval  is  due  to  the  random  occurrence  of  aircraft  calendar  downtime  in  the 
evaluation  data  set.  The  downtimes  also  have  a  large  variance.  It  would  improve  the  variance  if 
the  metric  were  changed  to  flight  hours.  Such  a  conversion  of  metrics  is  outside  the  scope  of  this 
evaluation. 

There  is  some  evidence  of  a  useful  prognostics  capability  provided  that  Remaining 
Useful  Fife  (RUF)  data  can  be  reliably  estimated  from  the  indicators,  but  this  is  beyond  the 
scope  of  this  study. 
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I.  CFAR  Autotrend  Discovered  Defects 


There  were  28  events  identified  by  the  CFAR  Auto  trend  that  indicated  a  possible 
defect.  All  but  four  detections  were  confirmed  by  the  DA  Form  2410  data;  these  four  cases 
represent  a  level  of  uncertainty  in  the  performance  analysis. 

The  perfonnance  analysis  produced  the  following  Key  Performance  Indicators  (KPIs) 
as  shown  in  Table  6. 

Table  6.  ADS-79  Metrics  for  CFAR  Autotrend  Using  DA  Form  2410  data 


Metric 

Fixed  Threshold 

CFAR  Autotrend 

TP 

2 

24 

FP 

0 

0 

FN 

22 

0 

The  majority  of  the  fixed  threshold  FNs  discovered  by  the  CFAR  occured  because  the 
fixed  thresholds  were  set  artificially  high  to  ensure  an  acceptable  FP  rate.  Because  this  is  a 
fielded  system,  these  defects  were  not  alerted  to  maintenance;  therefore,  it  is  considered 
reasonable  that  they  are  reported  as  fixed  threshold  FNs. 

Combining  the  two  sets  of  results  with  the  uncertainty  caused  by  the  undetennined 
detections  produces  a  best  case  as  shown  in  Table  7. 

Table  7.  Best-  and  Worst-Case  Comparison  of  CFAR  Autotrend  ADS-79  Metrics 


Metric 

Fixed 

Threshold 

CFAR  Autotrend 
Best  Case 

CFAR  Autotrend 
Worst  Case 

TP 

9 

37 

33 

FP 

1 

0 

4 

FN 

27 

0 

0 

The  FN  rate  for  CFAR  is  idealistic.  The  total  population  of  defects  has  been  discovered 
by  the  two  alert  detection  technologies,  not  selected  from  an  independent  ground  truth  set  of 
defects.  This  analysis  is  sufficient  to  compare  the  two  techniques,  but  it  does  not  provide  the 
absolute  performance  measure  for  the  CFAR  Autotrend  FN  rate. 

Overall,  there  were  411  CFAR  detections — a  very  large  number.  The  data  set  has  an 
artificially  large  number  of  events  embedded  in  it,  and  each  event  triggered  multiple  detections 
in  the  CIs  in  the  CFAR  Autotrend  processing.  These  detections  were  analyzed  as  true  or  false. 
There  were  1 1  detections  that  could  not  be  resolved,  which  represents  a  level  of  uncertainty  in 
the  analysis.  The  best-  and  worst-case  data  are  shown  in  Table  8. 

Table  8.  Best-  and  Worst-Case  Comparison  for  CFAR  ADS-79  Metrics 


Metric 

Best  Case 

Worst  Case 

TP 

378 

367 

FP 

5 

33 

Percent  of  FP 

8 

11 

15 


The  FPs  are  within  the  target  of  10  percent  in  the  best  case,  but  they  are  slightly  adrift 
in  the  worst  case.  Given  that  most  of  the  uncertain  CFAR  Autotrend  detections  have  been 
resolved  as  true  so  far,  it  is  unlikely  that  the  worst  case  would  materialize. 

The  timelines  of  the  CFAR  Autotrend,  when  compared  to  fixed  threshold  detections, 
are  shown  in  Table  9. 

Table  9.  CFAR  and  Autotrend  Detection  as  a  Function  of  Time 


CFAR 

Autotrend 

Data  Points 

-17 

-18 

Time  (days) 

-27 

-15 

The  inconsistent  distribution  of  time  versus  points  is  due  to  the  large  intervals  of  non¬ 
flying  times  that  are  randomly  distributed  through  the  data  set. 

The  number  of  alerts  in  the  CFAR  Autotrend  detection  relating  to  a  single  maintenance 
event  is  shown  in  Table  10. 


Table  10.  Number  of  Alerts  per  Maintenance  Event 


Number  of  Alerts/Event 

1 

2-3 

4-10 

10+ 

Number  of  Events 

10 

12 

13 

2 

From  Table  10,  73  percent  of  maintenance  events  resulted  in  more  than  one  alert  being 
generated.  These  results  correlate  to  the  number  of  CIs  used  on  each  component.  If  there  is  a 
rich  set  of  CIs,  there  will  be  a  rich  set  of  alerts  that  could  support  the  development  of  robust 
health  indicators.  It  is  noted  that  CIs  are  not  allocated  in  a  constant  way  across  the  components 
in  the  evaluation  database.  Consistency  in  the  application  of  the  technology  will  be  required  to 
provide  constant  health  indicators. 

J.  Analysis  of  Results 

PD  for  the  CFAR  Auto  trend  based  on  the  valid  test  cases  was  100  percent.  Test  cases 
included  a  number  of  faults  present  on  installation.  CFAR  Autotrend  is  designed  to  identify  a 
relative  change  in  the  signal  level  and  is  not  a  suitable  technology  for  detecting  installed  faults. 

The  results  clearly  demonstrate  that  the  ADS-79  False  Alert  target  of  10  percent  can  be 
met.  This  ADS-79  performance  metric  has  been  achieved.  The  known  false  alert  in  the  test 
cases  was  not  alerted  by  CFAR  Autotrend. 

The  automatic  production  of  thresholds  resulted  in  a  large  number  of  defect  discoveries 
in  the  data  set.  This  was  due  to  the  MSPU  gear  thresholds  not  being  set.  The  MSPU  is 
technology  in  service,  and  therefore  it  is  reasonable  to  say  that  these  discoveries  amount  to  the 
FN  rate.  Due  to  the  design  of  the  evaluation — all  faults  were  discovered  by  one  or  the  other  of 
the  techniques — there  were  no  identified  FNs  for  the  CFAR  Autotrend.  To  identify  the  true  FN 
rate  a  test  case  set  needs  to  be  constructed  from  the  DA  Form  2410  records  without  reference  to 
the  defect  detection  source,  and  the  perfonnance  of  alert  technology  must  be  evaluated. 
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The  total  number  of  identified  defects  was  33,  a  reasonable  sample  for  determining  the 
relative  detection  performance  between  the  techniques. 

There  is  evidence  of  prognostics  capability  as  shown  in  Figure  8. 


Level  Alert:  00-00138  7  TR  Drive  Shaft  (Params:  7;7;6) 


Trend  Alert:  00-00138  7  TR  Drive  Shaft  (Params:  2.3;60) 


Figure  8.  Demonstration  of  Prognostic  Capability 

From  the  plots  in  Figure  8,  it  can  be  seen  that  a  clear  level  detection  did  not  result  in 
immediate  maintenance  intervention.  There  is  a  clear  trend  detected  following  the  level  alert. 
This  results  in  a  change  in  level  to  a  point  where  the  component  was  probably  removed,  which 
represents  a  clear  prognostic  opportunity.  In  this  circumstance,  what  would  be  the  criteria  for 
removing  the  component? 

There  are  a  large  number  of  double  step  level  changes  as  shown  in  Figure  9.  Some 
ambiguity  exists  because  the  initial  step  may  be  either  the  incipient  defect  or  the  installation  of  a 
new  component.  There  is  a  need  for  a  reset  on  the  installation  feature  in  the  system  to  capture 
installation  events  and  prevent  this  ambiguity;  otherwise,  using  this  data  as  a  prognostics  alert 
would  not  be  possible. 
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Level  Alert:  00-00151  5  TR  Drive  Shaft  (Params:  7;7;6) 


Level  Alert:  00-00151  6  General  (Params:  7;7;6) 


14.87 

12.87 

10.87 

8.87 

6.87 

4.87 

-460  -420  -380  -340  -300  -260  -220  -180  -140  -100  -60  -20 


Figure  9.  Double  Step  Example 

K.  Independent  Validation 

Apart  from  the  primary  evaluation,  AMRDEC’s  Aviation  Engineering  Directorate 
(AED)  Aeromechanics  Division  further  perfonned  an  independent  evaluation  of  the  features  and 
performance  of  the  Humaware  software.  This  evaluation  examined  the  functionality  and 
simplicity  of  setting  the  various  CFAR  Autotrend  parameters  for  the  purpose  of  optimizing  the 
results.  The  evaluation  also  extended  the  test  set  of  data  to  include  multiple  new  data  sets 
containing  known  component  failures.  CFAR  Autotrend  was  able  to  accurately  detect  100 
percent  of  the  known  failures,  including  one  failure  that  the  fixed  thresholds  were  unable  to 
detect.  A  summary  is  shown  in  Table  11. 


Table  11.  Independent  Evaluation  Summary  of  Ground  Truth  Data 


Metric 

Fixed  Threshold 

CFAR 

TP 

8.5 

10 

FP 

0 

0 

FN 

1 

0 
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Including  the  known  failures,  CFAR  provided  135  alerts  of  possible  events,  whether 
they  were  simply  maintenance  events  or  impending  component  failures.  By  researching  the  DA 
Form  2410  reports  for  the  appropriate  aircraft,  AMRDEC  was  able  to  successfully  link  76  of  the 
alerts  to  known  maintenance  events.  There  were  58  alerts  that  could  not  be  successfully 
correlated  to  recorded  maintenance  events;  however,  many  of  these  were  related  to  non-2410 
recorded  components,  such  as  drive  shaft  and  tail  rotor  imbalances.  This  is  summarized  in 
Table  12. 


Table  12.  Total  CFAR  Detections 


Metric 

CFAR 

TP 

76 

FP 

1 

Unknown 

58 

Total 

135 

In  terms  of  events,  the  perfonnance  is  shown  in  Table  13. 


Table  13.  CFAR  Performance 


Metric 

CFAR 

TP 

22 

FP 

1 

Unknown 

12 

Total 

35 

Much  of  the  work  needed  to  optimize  the  parameters  was  performed  during  the  primary 
evaluation,  and  the  results  were  used  as  the  base  parameter  set  for  the  independent  validation. 
AMRDEC  experimented  slightly  with  tweaking  the  parameters  and  was  able  to  obtain  a  better 
knowledge  of  how  such  methods  affect  the  resulting  alerts.  For  whatever  reason,  the  Autotrend 
portion  of  the  software  did  not  function  properly  on  the  independent  computer  so  there  are  no 
results,  either  positive  or  negative,  to  show  for  it  in  this  independent  evaluation.  Due  to  time 
constraints,  AMRDEC  was  unable  to  fully  explore  the  use  of  setting  bull’s-eye  parameters  in 
CFAR.  The  function  was  explored  during  the  initial  training  of  the  software,  however,  and 
would  likely  play  a  vital  role  in  many,  if  not  all,  of  the  CIs  were  this  software  to  be  utilized  in  the 
near  future. 

L.  Findings 

The  principal  findings  from  the  evaluation  program  are  that  CFAR  Autotrend — 

•  Exceeds  the  fixed  threshold  detection  accuracy. 

•  Provides  a  significantly  earlier  detection  than  the  fixed  thresholds. 

•  Detects  maintenance  events  (material  faults)  that  are  not  identified  by  fixed 
thresholds. 

•  Is  ready  for  use  as  an  engineering  tool  in  its  current  fonn. 
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III.  RECOMMENDATIONS 


The  assessment  finding  is  that  the  technology  is  ready  for  the  next  stage  of  development. 
This  section  aims  to  identify  what  these  steps  are.  There  are  three  areas  of  development.  The 
first  is  to  identify  the  steps  needed  to  develop  an  in-service  CFAR  Autotrend  alert  detection 
capability.  The  second  is  to  develop  and  execute  the  service  delivery.  The  third  is  to  identify  the 
improvements  in  diagnostics  and  prognostics  performance  made  possible  by  the  technology. 

A.  Develop  In-Service  Capability 

The  following  work  items  represent  the  issues  that  need  to  be  addressed  by  AMRDEC 
to  develop  an  in-service  CFAR  Autotrend  capability: 

•  Full  verification  for  all  relevant  Cl  data  sets  for  HUMS  equipped  fleets.  (This  is 
required  to  ensure  that  the  performance  reported  in  the  assessment  is  valid  for 
the  whole  fleet  data  and  that  the  parameters  are  correctly  set  and  allocated  to 
the  Cl  data  types.) 

•  Verify  the  levels  of  conservatism  to  be  applied  to  the  setup  parameters  to 
minimize  the  No  Fault  Found  (NFF)  rate  and  not  compromise  the  TP  and  FN 
rates.  (The  assessment  shows  that  the  processing  can  be  too  sensitive  to  defects, 
raising  the  prospect  of  increasing  the  NFF  rate  and  costing  useful  life.  The 
CFAR  Autotrend  setup  parameters  can  be  backed  off  to  reduce  sensitivity.) 

•  Assessment  for  other  data  sets,  such  as  engine  Health  Indication  Test  (HIT) 
check  trending.  (There  are  other  data  sets  being  recorded  in  the  HUMS,  engine 
performance,  and  usage  that  could  benefit  from  the  use  of  the  technology.  This 
should  be  evaluated  before  fielding.) 

•  Verify  integration  with  existing  red  static  alert  thresholds  in  the  CFAR 
Autotrend  using  the  Binary  or  BulFs-eye  Alert  feature.  (It  will  be  necessary  to 
validate  the  performance  of  the  unified  system  of fixed  “red”  alerts  combined 
with  the  dynamic  “yellow”  warning  alerts.) 

•  Revise  the  training  and  doctrine  for  the  use  of  trend-based  alerts  by  personnel 
utilizing  the  system.  (Having  trend  alerts  available  is  a  new  capability;  correct 
usage  needs  to  be  determined  before  fielding.) 

•  Validate  the  software  functionality  and  the  operational  interfaces  for  the  control 
and  processing  of  the  Cl  data.  (The  current  software  is  configured  for 
application  development  work  and  verifying  performance.  The  in-service 
application  will  require  different  functionality  for  data  management,  that  is,  the 
processing  of  data  in  slices  as  they  are  downloaded  from  the  aircraft.  The  data 
output  formats  for  diagnostics  and  prognostics  displays  and  applications  need 
to  be  implemented.  All  of  this  functionality  is  outside  the  proprietary  package 
and  is  implemented  in  the  “connectors.  ”  This  functionality  will  need  to  be 
developed  and  validated.) 

•  Develop  a  maintenance  action  reset  facility.  (This  has  been  identified  as  a 
feature  necessary  to  prevent  maintenance  action  from  being  alerted  as  defects.) 
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•  (Optional)  Improve  installed  defect  detection  capability.  (Investigate  the  use  of 
trends  to  improve  performance  of  installed  defect  detection  performance.) 

•  Develop  DO  178  B  (Level  D)  certifiable  version  of  core  CFAR  Auto  trend 
processing.  (The  certification  standards  applied  to  the  software  need  to  be 
consistent  with  the  other  processes  in  the  diagnostics  data  flow  in  order  to  grant 
maintenance  credits.) 

B.  Service  Delivery 

The  service  delivery  includes  all  the  program  requirements  necessary  to  field  the 
capability. 

This  will  be  an  independent  activity  for  each  DSC  type. 

CFAR  Autotrend  processing  needs  to  be  applied  where  the  Cl  data  is  first  reviewed  in 
order  to  gain  the  performance  benefits.  One  may  argue  that  this  should  be  done  on  either  the 
aircraft  system  or  on  the  ground  stations  (so  that  all  users  see  the  same  alerts).  For  the  purpose 
of  this  work  breakdown,  it  is  assumed  to  be  a  ground  station  application.  The  architecture  is  for 
three  software  modules — an  Input  Connector,  an  Output  Connector,  and  a  standard  CFAR 
Autotrend  processing  module.  The  following  applies  for  each  application: 

•  Develop  interface  control  documents,  functional  specifications,  test  and  fielding 
plans. 

•  Develop  Input  Connector  to  support  access  and  management  of  data  inputs. 

•  Develop  Output  Connector  to  support  interfaces  with  diagnostics  screens  and 
processes. 

•  Integrate  and  test  the  three  modules. 

•  Integrate  in  the  hardware.  (This  will  require  DO-1 78B  (level  D)  recertification 
of  the  entire  process  in  the  ground  station.) 

•  Implement  the  existing  red  static  alert  thresholds  in  the  CFAR  Autotrend  using 
the  Binary  or  Bull’s-eye  Alert  feature. 

•  Validate  upgrade  in  ground  stations. 

•  Modify  handbooks  to  provide  operating  procedures  for  trend  alerts. 

•  Execute  fielding  plan.  (This  is  a  data-driven  process  so  there  will  be  the  need 
to  develop  a  system  for  AMRDEC  to  monitor  and  manage  data  quality.) 
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C.  Diagnostic  Processing  Improvement 

The  assessment  program  demonstrated  that: 

•  The  CFAR  Autotrend  technology  can  provide  a  tool  for  data  discovery  that  can 
support  the  Verification  and  Validation  (V&V)  of  diagnostics  and  possibly 
prognostics  processes. 

•  The  alerts  have  attributes  that  can  improve  specificity  of  the  diagnostics.  This  is 
necessary  to  support  the  certification  of  maintenance  credits. 

•  The  alerts  have  demonstrated  that  a  significant  RUL  can  be  achieved.  For 
prognostics  to  be  certified,  this  must  be  verified  as  being  a  safe  life  remaining. 

The  following  work  items  relate  to  developing  a  diagnostics  and  prognostics  capability 
based  on  using  the  alert  detection  capability  of  CFAR  Auto  trend: 

•  Diagnostic  Development  Integration  into  engineering  support  systems. 

(Develop  the  input  and  output  connectors  for  the  AMRDEC  installation  to  be 
applied  to  data  warehouse  and  other  data  repositories,  and  to  report  results  to 
other  analysis  packages.) 

•  Reevaluate  the  extant  Cl  performance  review  to  determine  if  CFAR  Autotrend 
technology  improves  the  AMRDEC  batting  averages. 

•  Use  CFAR  Autotrend  to  discover  the  defects  in  Data  Warehouse  data,  and 
develop  Golden  Database  to  support  diagnostic  development  V&V. 

•  Statistically  evaluate  level  alerts  from  the  Golden  Database  to  determine  if 
re-basing  levels  to  the  primary  threshold  values  can  improve  diagnostic 
discrimination  of  defects  that  compromise  airworthiness  criteria  from 
acceptable  wear. 

•  Statistically  evaluate  if  box  trend  rates  from  the  Auto  trend  processing  can 
improve  diagnostic  discrimination  of  defects  that  compromise  airworthiness 
criteria  from  acceptable  wear. 

•  Develop  Bayesian  probabilities  and  combinational  rules  for  Health  Indicators 
utilizing  the  results  of  3.3.1  and  3.3.2  to  develop  criteria  for  safe  RUL.  (This 
will  require  the  allocation  of  Cl  types  to  defects  to  be  rationalized  and  applied 
more  uniformly  in  order  that  the  Bayesian  probabilities  can  be  calculated.) 

•  Modify  CFAR  Autotrend  alert  detector  parameters  to  implement  the  improved 
diagnostics  and  prognostics  alert  capabilities. 

•  Verily  performance  on  Golden  Database. 

•  Develop,  as  appropriate,  Cl  diagnostics  knowledge  base  to  include  heuristic 
diagnostics  capabilities  from  sources  other  than  the  DSC. 

•  Develop  a  Service  Delivery  Plan. 
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LIST  OF  ACRONYMS  AND  ABBREVIATIONS 


AMRDEC 

Aviation  and  Missile  Research,  Development,  and  Engineering  Center 

APU 

Auxiliary  Power  Unit 

CFAR 

Constant  False  Alarm  Rate 

Cl 

Condition  Indicator 

CSV 

Comma-Separated  Value 

DAD 

Detection  Algorithm  Development 

DSC 

Digital  Source  Collector 

FAR 

False  Alarm  Rate 

FN 

False  Negative 

FP 

False  Positive 

HIT 

Health  Indication  Test 

HUMS 

Health  and  Usage  Monitoring  System 

ID 

Identity 

KPI 

Key  Performance  Indicator 

MooN 

M-out-of-N 

MSPU 

Modernized  Signal  Processor  Unit 

NFF 

No  Fault  Found 

PAT 

Parameter  Allocation  Table 

PD 

Probability  of  Detection 

PTE 

Primary  Threshold  Exceedance 

RUL 

Remaining  Useful  Life 

TiN 

Trend  in  Noise 

TN 

True  Negative 

TP 

True  Positive 

UK 

United  Kingdom 

v&v 

Verification  and  Validation 
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KEY  TO  CHARTS 


The  results  of  the  Constant  False  Alarm  Rate  (CFAR)  Autotrend  processing  are  displayed 
as  graphical  reports. 


CFAR  Processing 


Level  Alert  Imported  Data:  Sensor  T67-CI110;8;7) 


Autotrend  Processing 


Trend  Alert  Imported  Data:  Sensor  FR037-CI2  (Params:  2;cont) 


A  key  for  these  reports  is  shown  below. 

CFAR  Autotrend 


- Value 

- Value 

- Box  T rend 

- Primary 

Threshold 

Car  Trend 

Secondary 

Threshold 

Threshold 

T  rend 
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NOTE:  For  the  CFAR,  the  Secondary  Threshold  is  not  a  separate  plot.  The  Secondary 
Threshold  identifies  where  the  secondary  processing  criterion  has  been  satisfied  and  therefore  an 
alert  detected.  This  “Threshold”  is  indicated  by  plotting  the  data  steam  values  in  yellow. 

Chart  Headers 
CFAR 


Level  Alert:  00-00141  SF-N1-16  31  Gear  &  Bearing  (Params:  7;7.;6) 


Autotrend 


N 


M 


Trend  Alert:  00-00130  4 


GG  Shaft  (Params:  2 .3; 60) 


Maximum 
Car  Length 
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